NASA Contractor Report 181663 
ICASE REPORT NO. 88-25 


ICASE 


ESTIMATION OF TIME- AND STATE-DEPENDENT DELAYS AND 

OTHER PARAMETERS IN FUNCTIONAL DIFFERENTIAL EQUATIONS 

( NASA-CB- 1 816 63) ESTIMATION CF TIME- AND N88-2<4 164 I 

STATE-DEPENDENT DEXAYS AND CTEER' PARAMETERS 

IE FUNCTIONAL DIFFERENTIA! ECU AUG NS Final 

Report (NASA) 53 p CSCI 12A Unclas 

G3/59 0145737 

: III J 


K. A. Murphy 


Contract No. NAS1-18107 
April 1988 


INSTITUTE FOR COMPUTER APPLICATIONS IN SCIENCE AND ENGINEERING 
NASA Langley Research Center, Hampton, Virginia 23665 

-> 

Operated by the Universities Space Research Association 


NASA 

National Aeronautics and 
Space Administration 

Langley Research Center 

Hampton, Virginia 23665 



ESTIMATION OF TIME- AND STATE-DEPENDENT DELAYS 
AND OTHER PARAMETERS IN FUNCTIONAL DIFFERENTIAL EQUATIONS 


K.A. Murphy 

Department of Mathematics 
University of North Carolina 
Chapel Hill, NC 27514 
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condition) appearing within a nonlinear nonautonomous functional differential 
equation. The original infinite dimensional differential equation is approximated 
using linear splines, which are allowed to move with the variable delay. The 
variable delays are approximated using linear splines as well. The approximation 
scheme produces a system of ordinary differential equations with nice computational 
properties. The unknown parameters are estimated within the approximating systems 
by minimizing a least-squares fit-to-data criterion. We prove convergence theorems 
for time-dependent delays and state-dependent delays within two classes, which say 
essentially that fitting the data by using approximations will, in the limit, provide a 
fit to the data using the original system. We present numerical test examples which 
illustrate the method for all types of delay. 
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1. Introduction. Delay and functional differential equations (FDEs) can be used to 
model a variety of phenomena in the physical and natural sciences. Our focus here 
is on models in which the delays are nonconstant: They are either time- or state- 

dependent. 

A problem from electrodynamics involving state-dependent delays was 
considered by Driver (e.g., [7] and [8]). In [8] he derives and analyzes systems of 
equations of the general form: 

y^t) = f^t.yCt), y(g 2 (t,y(t))), .... y(g m (t,y(t)))) (i-l,...,n). 

Such equations can be used (as in [7], for example) to describe the motion of two 

electrically charged particles. In such a case, the delays are state-dependent 
because the effect of one particle on another is not felt until the emitted electrical 
signal has traveled the distance between the two particles; the time lag depends on 
the distance, which, in turn, depends on the positions, which is (part of) the state of 
the system. 

Another application area rich in FDE models is biology. In [10], for example, 
a model for an antigen-antibody interaction is discussed, involving a system of 
functional differential equations. The full model incorporates thresholding effects, 
which introduces two delays. The model equations are of the form 

x(t) = f(x(t),x(to(t))) where x(t) represents the (vector) state of the system, and 
each retarding function to satisfies an equation of the form: 

f g(t — s, x(s) ) ds = 0. 

J to(t) 

Here, the function to depends explicitly only on time, but is also state-dependent 
through the above integral equation. 

A second example from biology arises in the modeling of insect population 
growth. Consider an insect which passes through several larval stages (called 
’’instars”) during the course of its lifetime. In [17], a model for the population 
growth of such insects is developed under the assumption that the passage from one 
instar to the next is contingent upon a certain body weight gain. An alternative 
assumption (see, e.g., [13]) is that this passage is determined by the exposure of the 
insect to a sufficient number of degree-days. In this case, one can begin in an 
analogous way to Nisbet and Gurney in [17], but the final model developed is a 
system of delay differential equations with time-dependent delays (this model 
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development will be the subject of a future paper). 

Many researchers have investigated the problem of approximating functional 
differential equations for the purpose of numerical solution (the ’’forward problem”) 
and also the related problem of estimating unknown parameters, particularly delays, 
occuring within FDEs (the ’’inverse” problem). As an example of the former, we 
mention the work of Feldstein, et. al. (see, e.g., [9], [15], [16]). This work consists of 
developing accurate, high order methods for the forward problem; they consider, in 
particular, delay equations in which the delay is state-dependent. One approach to 
the numerical solution of delay equations is to begin with methods developed for 
ordinary differential equations, with the addition of an interpolation procedure for 
the approximation of solution values at lagged times. In the references cited above, 
efforts focus on determining locations of discontinuities in derivatives of the 
solution, and exploiting this information in order to retain a higher degree of 
accuracy in the numerical solution. 

In this paper, we are concerned with the parameter estimation problem. In all 
but a few special cases, one expects to use an iterative (on the unknown parameters) 
method combined with some numerical approximation scheme for solving the 
differential equation. In the inverse problem, accuracy is important to some extent, 
however, as one is in general trying to fit field data, a high degree of accuracy is 
not as important as a fast and efficient solution algorithm. In the work of Banks, 
et. al., (see, e.g., [1], [2], [3]), one can find various approximation schemes (involving 
the use of splines and the ’’averaging” method); the differential equations are posed 
in an abstract, operator-theoretic setting, within which convergence results are 
obtained (i.e., parameters estimated using the approximations converge 
(subsequentially) to parameters which provide a fit for the original model 
equations). These results pertain to the estimation of multiple constant delays (and 
other parameters) in a system of functional differential equations. 

In the present paper, we extend the ideas developed in [1] - [3] (primarily [3]), 
in order to devise an approximation scheme and prove convergence results for the 
estimation of a nonconstant delay in a nonautonomous, nonlinear functional 
differential equation. The generalization to a system of equations is straight- 
forward, and is not considered here. We consider the estimation of only a single 
delay, although it is expected that the estimation of multiple delays should also be 
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a straightforward modification. 

In our presentation here, we begin in section 2 by describing the type of 
model equations in which we are interested and state the parameter estimation 
problem precisely. In section 3, we formulate the differential equation abstractly 
and develop the approximation schemes for the states and the variable delay. We 
state and prove convergence theorems in section 4, discuss the numerical 
implementation of the scheme in section S, and conclude with some numerical test 
examples in section 6. 


2. The Parameter Estimation Problem. We formulate our parameter estimation 
problem as follows. We will assume that we are interested in modeling some 
phenomenon for which we have a general form of model equation, but some unknown 
parameters appear in the equations. We assume further that we have observed our 
system, and collected data. We then wish to determine the unknown parameters by 
fitting the model equation to the data. Here, our model equation is a functional 
differential equation of the general form: 


( 2 . 1 ) 


x(t) = f(t,7,x(t),x(t-T),x t ) 0 ^ t ^ T 

X 0 (s) = 0(s) — T| uo ^ S ^ 0 


where x(t) 6 R, x t denotes the function x t : s -» x t (s) = x(t -f- s) where s varies 
over [-r, 0] (unless this interval is otherwise specified), and 7 represents a vector 
of parameters occuring in the model equation (with perhaps some functional 
components). The unknowns in this equation might be any of 7» T, or <f>. If the 
delay is explicitly (and solely) time-dependent, it is the function r:t-»r(t) which 
we assume is unknown and to be estimated. We shall also consider two classes of 
state-dependent delays. In the first class, we assume the delay is determined 

r i 

through a relationship of the form I g(x(s))ds = 0; here it is the function g 

J t-r(t) 

and the constant 0 which we shall assume are unknown and to be estimated. In the 
second class, we assume the delay is of the form r(t) = g(x(t)), and it is the 
function g:x-*g(x) which we wish to determine. 

Suppose now that our data is given by {Xj)|!Lp where Xj corresponds to 
the solution of (2.1), for some value of the parameters, evaluated at tj, i.e., x(t|). 
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We would like to determine the unknowns in (2.1) such that the least squares fit-to- 
data criterion: 

m 

J(p) = ZlxCtj; P ) — Xjl 2 

i-1 

is minimized. We use p to represent the unknown parameters and we write x(t; p) 
to emphasize that the solution to (2.1) depends on these parameters; in the time- 
dependent delay case, p — (7 >T,0), and in the state-dependent delay cases, 

P = (7,g or p — 

We shall assume that the unknown parameters lie in some constraint set, 
denoted by II = QXTXJ, for time-dependent delay, or II = QXTgXJ for state- 
dependent delays of the first class, or II = QXTgXJ for the second class, where 
7£Q» r ET, <f>E5 , (g,0)€Tg = TgXO and gETg. In the state-dependent delay 

case, it will be convenient to define the constraint set T g in such a way that the 
resulting r, as a function of time, will belong to the same constraint set T which 
is defined for the time-dependent delay. 

First consider the unknown parameters represented by 7* This will, in 
general, be a vector of unknowns, each component lying in R, or in some function 
space. For definiteness, let us suppose that 7 consists of a vector of u 
constants and one functional unknown, so that 7 belongs to R^XCIO/T). The set 
G will represent some appropriately chosen compact subset of R^XCIO.T), for 
example, we might take 

G = { (7i, 7 2 ,-.-,7/„ 7 0 ) G TR^XHHO.T) I for i = 1,...,// and 

I7olo© ^ c 0 , I7 0 | 0 <1 do } 

Using the Ascoli Theorem, one can argue that such a choice satisfies 
(HG) The set G is a compact subset of R^XCtOJ). 


We define the sets associated with the delay as 
T = {tE W 1,co (0,T) | T 0 £T£r, Itl^/z a.e. tE[0,T]} 

T g m {gEW 1>00 (x,x) | g L £ g ^g u , |g(Xi) — g(x 2 )|^/i g |xi — x 2 l > and 
0 s {0ER 10^0^0} 

where t 0 , r, g L , g u , 0, 0, pi, fig are positive constants which are either known or 
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chosen a priori. The constants x and x represent lower and upper bounds (perhaps 
only estimates) on the solution. Note that theoretically we could consider g as a 
function defined on (- 00 , co), however when we consider approximating functions for 
g, it is necessary for our approximation schemes to consider functions defined on a 
finite interval. We can argue, using properties of our model equations and 
parameters that all solutions will be bounded, uniformly in the parameter sets. 
Thus, there do exist numbers x and x such that we can treat g as being defined 
only on the interval [x,x]. We shall say more later about the choice of these 
numbers in practice. Again, one can argue using the Ascoli Theorem that each of 
the sets defined above satisfy 

The set T is a compact subset of C(0,T). 

(HT) The set Tg is a compact subset of C(x,x), and the set T g is a 

compact subset of C(x,x)XR. 


Finally, we will assume the unknown initial data lies within a set 5, which 
is a subset of H^-r.O). As we iterate on the unknown parameters (in particular, on 
r ), we will perform our computations using 0 as a function defined on [-r| t=0 , 0], 
and for a given T| uo ^ r, the delay equation (2.1) is well-posed for 0 defined on 
[ - T (t«o » 01. However, we can consider 0 as an element of HH-r.O) by continuously 
extending it. We assume that 

(HI) The set 3 is a compact subset of C(-r,0). 

We assume the function f is continuous in all its arguments and satisfies a 
Lipschitz condition of the following form: 

(HF) There is a positive function m(t) E L 2 (0,T) such that for any 7 £(j» 
and for any x ), ftafo’fe) € R X R X L 2 (-r, 0), it follows that 

I f(t, 7 t€ufi.i?i) - I £ m(t) { \u - e 2 l + I?1 - Cal + I»?1 - V 2 \ 0 } 

where | • | 0 denotes the norm in L 2 (-r,0). 

Several authors have investigated questions of existence and uniqueness of 
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solutions of equation (2.1) (see for example, [8] and the references therein, [11], or 
[20]). They make varying assumptions on the function f and the delay (s). The 
assumptions we have made above are sufficient to prove the convergence arguments 
which are crucial to the parameter estimation problem; this is the problem of 
concern to us here. 

We note that, as with any inverse problem, questions of well-posedness arise, 
i.e., one questions whether there is a unique solution to the parameter estimation 
problem (in fact, as will be seen in the final section, examples are easily constructed 
for which there is not a unique solution unless the problem is slightly 
reformulated), and how these solutions depend on the data. These are important 
questions but are outside the scope of this paper. 


3. Abstract Formulation and Approximations. We shall begin by rewriting 
equation (2.1) in abstract form. This provides a framework for our approximation 
theory and facilitates our convergence arguments. For convenience in notation, let 
us define Q. = QXT and let q-(7»T); it is to be understood that in the case of 
state-dependent delay, r(-): t*-*r(t) represents the delay obtained from the unknown 
parameters g ,0 or g. Suppose x(t) represents the solution of equation (2.1), 
defined for all t£[0,T], given q£Q. and initial condition 0 G3. Just as in [2] 

and [3], for the constant delay problem, we shall take our state to be z(t) = 



0 ( 0 ) 

0(0 

considering x t (s) defined for all -r<;s^0. We could alternatively, and more 
economically, define our state space to be X(r(t)) = R X L 2 (-T(t),0). This is a state 
space which we shall use later, but it is more convenient to consider our ’’true 
solution” to be in the stationary state space, Z. Given that x is a solution of 
(2.1), it follows that z is a solution of the abstract ordinary differential equation: 


with state space Z = R X L 2 (-r,0), and we define 0 = 



Here we are 


(3.1) 


z(t) = _4.(t;q) z(t) 

z(0) = 0 


where A 


is the nonlinear operator defined by 


-4.(t;q) 


Zo 

1 

F(t,q,z 0 ,z) 

z(.) 

Dz 


with 
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we are using D to represent 


aB = dom^l(q) = {^ 0 ) j G RXHH-r.O) | z(0) = z o }; 
the differentiation operator and we define F(t,q,z 0 ,z) = f(t,7»Zo»z(-T(t)),z). 


It should be noted that if we had chosen to define z(t) in X(r) rather 
than in Z, this equation would have been written identically, except for a slight 
change in the definition of dom.4. As it will be necessary to consider functions in 
both state spaces Z and X(r), we now define extension and restriction operators. 


Fix rGf. Given z 


[ 

z(-) 


GZ, let t(r)z be the element of X(r) obtained by 


restricting the function zGL 2 (-r,0) to be a function in L 2 (-t, 0). Similarly, if 


\ y 0 
y(-) 

t - 


GX(r), we define e(r)y to be the element of Z obtained by extending y 


in a constant, continuous way. 


Equations (2.1) and (3.1) are equivalent in the sense that if x is a solution 
of equation (2.1), then z(t) = (x(t), x t ) T (with x t defined on [-r, 0] ) is a solution of 
equation (3.1), and, conversely, if z is a solution of (3.1), then i(t)z can be 
identified with (x(t), x t ) T with x t defined on t-T,0], where x is a solution of (2.1) 
(this equivalence can be argued rigorously just as in [3] ). Thus, this equivalence 
can be used to argue that whenever z(0)G2B, then z(t)G2B for all tG[0,T]. 


We shall use the following notation for norms. Absolute value will be 
denoted by | • |, the norm in L 2 (-r,0) by | • | 0 , and the norm in Z by || • || 0 . 

Given yGL 2 (-r(t),0), we define the norm by |y | Z T — I y 2 ds. The norm on X(r) 

J -Ttt) 

is the usual cross product definition, and will be designated || • || T . 

We now formulate the parameter .estimation problem in terms of the abstract 
equation (3.1) as follows: 


(P) 


m . 

Min J(p) = | z 0 (t i ;>o) — Xi I subject to 

/>GII i-i 1 1 


z(t;p) 


z 0 (t \p) 
z(t, • ;p) 


a solution of equation (3.1). 


Problem (P) is clearly an infinite dimensional problem, in that the evaluation of z 0 , 
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and hence of J, requires the solution of the infinite dimensional equation (3.1). 
Thus, we shall approximate the spaces Z and X(r) by a sequence of finite 
dimensional approximating subspaces, and develop a sequence of corresponding 
approximating differential equations. 

Our choice of approximation scheme is motivated by that of 12] and [3], but 
we must modify these ideas to accomodate the time-varying delay. For each N, 

consider tET and fix t6[0,T]. Define the knot sequence (kj(r)}£ 0 by kj = . 

Let {B-(r)}[^ 0 be the linear spline basis elements, or ”hat” functions (see, e.g., [18]) 

defined on the r-dependent grid {kj(r)}" 0 . Let Bj = and X N (r) = span{Bj}[^ 0 . 

Notice that X N (r)CX(r), but X(r) is not technically a subset of dom^4.(t;q); thus we 
define X n (t) = e(r)X N , which is a subset of dom-4.(t;q). The sets X N (r) and X n (t) 
are both finite dimensional (of the same dimension), and from a computational point 
of view, they are equivalent. We can think of X N (r) as the span of {3^}, where we 
define == eBj. We define X N (r) for theoretical reasons, but we perform our 
computations using elements of X N (r). We define the orthogonal projection (in the 
X(r)-topology) for each N and tE’T by P N (r(t)): X(r(t)) -» X N (r(t)). We further define 
the operator T n (t) = €(t)P n (t)i(t), which is essentially equivalent to P N , except that 
it maps elements of Z to elements of X N C dom^4.(t;q). Let ^4. N (t;q) = 3* N (T)-4.(t;q). 
The approximating equations for (3.1) are given by 


(3.2) 


2 N (t) = -4 N (t;q) z N (t) 
z N (0) = T n (t(O))0 


where, for each t, z N (t) 6 X N (r(t)). 

One could formulate a corresponding parameter estimation problem, defined in 
terms of z N in place of z; there is still an aspect of infinite dimensionality, 
however, in that r (or g), and possibly 7 are unknown functions. We describe an 
approximation scheme for r or g (assuming for ease in presentation of our ideas 
that <j> and 7 are either simply unknown constants or have known a priori 
parametrizations involving only unknown constants; if this is not the case, then the 
ideas for r, g presented here can be easily applied to these parameters as well), and 
then state our parameter estimation problem in a completely numerically 
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implementable form. The ideas we use here for approximating the delay are similar 
to those described in [4], [5], [6] and [12] for the secondary approximations of 
variable parameters in partial differential equations. 

In the case where r = r(t) is the unknown, we proceed as follows to define 
the secondary approximations. Given T, the final time of interest in the original 

delay differential equation, define the knot sequence {/Cj}£ 0 by /Cj = and let 

{bj}^ 0 be the hat functions defined for this grid. Let S M =* span(bj). We define a 
sequence of finite dimensional sets by T M = TDS M . Let / M represent the 

interpolation operator from V* to S M (i.e., given t6T, I M r is the function in S M 
which satisfies (/ m t)(/Cj) — r(/cp for i=0,l,...,M). Then T M can be characterized as 
gr M = fY. Given that tGT, it follows from standard results in the theory of 
linear splines (see, e.g., [19]) that in C[0,T] as M-*oo. Recall (HT) states 

that the set T forms a compact subset of C[0,T], and each T M CT is also a compact 
subset of C[0,T] (this follows from the fact that T M is a closed subset of T). 

In either of the cases of state-dependent delay, one first needs to determine 
the interval [x,x] on which to define the approximations. One might begin by 
inspecting the data, and then choosing the lower and upper bounds accordingly; it is 
the experience of the author (both for the estimation of delays and also with the 
estimation of unknown state-dependent parameters in partial differential equations, 
[6]) that, in practice, it can be determined in the course of the estimation procedure 
that one has either over- or underestimated the size of this interval. The 
computer program can be written so that the interval is automatically extended if 
the approximate state z£ takes on a value outside the original interval. If the 
original interval is chosen to be too large, then one sees convergence of the delay 
on the interior of the interval, and ’’chattering” behavior around the endpoint(s) 
where one is trying to estimate a dependence where none is there to be seen. From 
a theoretical standpoint, one could equivalently consider the approximating 
functions, g M , as being defined on all of (- 00 , 00 ) by defining g M (x) = g M (x) for 
x £ x and g M (x) = g M (x) for x ;> x. 

Given the interval [x,x], define the knot sequence by K j = i(x-x)/M, 

and let {bj}^ 0 be the hat functions defined for this grid. Let S M = span{bj}, and 
Tg = T g PI S M . As above, we then let J M represent the interpolation operator from 
Tg to S M and notice that Tg can be characterized as Tg = ^Tg. We can argue 
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that for any gG^Tg I M g-»g in C[x,x], and that TgCTg is a compact subset of 
C[x,x]. We finally define ^Tg a Tg X 0. 

With *Tg, Tg 1 defined as above, let IT m QXtT M X3, n M ■ QX?£XJ, or 
IT s QXtTgXJ. Note that by assumptions (HG) and (HI) and per the discussion 
above, the sets II and II M are compact. We now state our approximate parameter 
estimation problem (for each N,M) as: 


(P 


N,M, 


Min J N (p) = 2l z o(ti»P) — Xi I 2 subject to 
pGlT ui 1 


z N (t;p) = 


z?(tsp) ’ 
z N (t, •;/>). 


a solution of equation (3.2). 


For each N,M, Problem (P N,M ) has a solution, p N,M GlI M , since each II M is compact and 
J N is a continuous function of / oGII m (this can be seen from the matrix-vector 
representations of (3.2) developed in a later section). Moreover, each p N,M is in II, 
also compact, and thus we conclude there is a convergent subsequence {p N|< ’ Mk } such 
that p —p with N^.M^-^oo and p Gil. It is our claim that this limit p is a 
solution of Problem (P). In order to verify this claim, we must first show that for 
an arbitrary convergent sequence of parameters in II, the states converge as well. 
This is established in the next section. 


4. Convergence Arguments. In this section we shall prove that a subsequence 
of the parameter estimates we generate by solving (P N,M ) converges to a solution of 
(P). As a first step, we shall demonstrate that convergence of an arbitrary sequence 
of parameters implies a corresponding convergence of solutions of the approximating 
differential equations to a solution of the original FDE. 

Before we can prove this convergence result, we first establish some 
preliminary results in the form of lemmas. Throughout this section, we let 

T = { TG W 1,o °(0,T) | 0<t o <. r(t) <, r VtE[0,T] and 
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Lemma L Suppose {r M } is an arbitrary collection of functions in T and 
z(t) 6 Z for each t 6 [0,T]. Then ||p n (t m )i(t m )z— i(t m )z||^ m -* 0 as N-»oo, for 
each t € [0,T]. 

Proof. Let us write P N,M = P N (r M ) and t M = t(r M ). We begin by assuming 
that for fixed t, z(t) G 3D. We shall use the following notation: Let 

z(t) = z = ^°)|, (notice that since zE3B, z 0 =z(0)), and let I N,M t M z = 

where z, N,M denotes the spline interpolant of the function z (actually, of its 
restriction, considered as a function in L 2 (-r M (t),0)), using the grid formed by 
subdividing the interval [-r M ,0] into N equal subintervals. 


For fixed t, we have 


<; - [zrw-zcojT + f (zT^-z) 2 as 

■* -r M (t) 

- f feT -= 

■* -T m ( t) 


-z) ds. 


Now, applying standard estimates (e.g., a slight modification of (2.14) in [19]) 
and using the fact that each r M belongs to T, we obtain 

|p w i m z-i m z|^.„ ^ feT f° (Dz(»)) 2 ds 
S [»&] 2 |_ r (Cz(t))2 ds - 

It is clear from the above estimate that fp N,M t M z— -♦ 0 as N -* o© for z G 3D 
and fixed t. Note that this convergence is independent of M as long as for each M, 
r M is in the set T. 


In fact the above convergence statement is also true for any z(t) € Z. This 
follows from the fact that 3D is dense in Z and that each p N,M t M j s linear and 
uniformly bounded (in N,M). □ 


We will often be interested in the X(r)-norm of functions that belong to the 
spaces Z, e(r)X(r), or £(t)X n (t) for some tGT. To be technically correct then, we 
should use the operator t(r) to first map the function into the space X(r). In order 
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to reduce the necessary notation, however, we will not write t when it should be 
clear from the context that it is needed. 

The next lemma tells how to relate the time derivative of a projected 
function to the projection of that function’s derivative. As indicated in an earlier 
discussion, the fact that these two elements are not equal is one respect in which 
the time- varying delay problem differs from the constant delay problem. 


Lemma 2. Assume r G 7 and z(t) 6 JD3 for each fixed t. Recall that 

•) ‘ 


epN _ g> N ( T ) — e(r) P n (t)i(t), let z(t)=|^°jj, and let us write P n (t)i(t)z = |^° Z , 


Then for any y£X N (r) with y= , we have for almost every t6[0,T]: 


« £(? N 2),y» T = « ? N g,y» T + [? • 


dt 1 


pO pO 

I s [Dz— D(P N z)j y ds + I (z— P N z)yds 

J-T J-t 


Proof. We first note that we define T N in terms of the extension operator 
e(r) so that all functions will be defined on the full interval [-r,0J; thus, while 
P N iz : [0,T] -* X(r(t)), and thus has a ’’moving range-space”, T N z : [0,T] -* X N CZ. 

We introduce the operator T N in order to make precise the time derivative -n-(T N z). 

at 

We recall that X N -eX N , X N is spanned by the set of basis functions {Bj}£_ Q with 


f$i(0)] 


and we shall write $- = eB^ = ^ ^ ^ 


GX n . All of this is a theoretical 


technicality; as discussed earlier, from a computational standpoint P N , X N , B^ and 

cpN, are e q U ivalent. 

For each t, P N iz is characterized by 

«P N tz , Bj » T = «z , » T for i— 0,1,. . .N 
or, equivalently, by 


(PqZ)Bj(O) + f (P N z) B^s = z o Bj(0) + [ zBj 
J-T J-t 


ds for i-0,l,...N. 


In terms of the extensions defined above, we can write 
(4.1) 

We now differentiate both sides of equation (4.1) with respect to t to obtain 


(Pgz)®^) + [ (eP N z)S i ds = ZoSjtO) + [ z^ds for i-0,l,...N. 
J-T J-T 
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rO T gcg 1 

-^(p^z)®^) + ^(eP N z)(s)® i (s) + (P N z)(s)-gJ(s) ds + T(t)((P N z)® i j 

• -Tit} J 

- ^ s i<°> + f + z w|<4= + • 

J -rctr J 

Because (eP N z)(t) and ®j(t) belong to W 1,o °[(0,T); HH-r.O)], the partials — 


and 


3®: 

are well defined (a.e. t). With a simple calculation, one can see 

dt 

3®i * 

that = -^s(D®j). After making this substitution in the above equation and 
integrating the corresponding terms by parts, we obtain 

(4.2) ^(P^z)®^) + | 3 - ^ ) ® i ds + i|| sD(P n z) ®j ds + | ® i (P N z)ds 

rO 


^Sj(O) + 


sDz®jds + ^B^zds 


J -T 


| ! V s + i 

J -T 

Because S^-Bi for s£[-t, 0], equation (4.2) is equivalent to 

-0 p0 

s D(P n z) Bj ds + B-(P N z) ds 

-T J-T 


((iwa.spv + ? 


f° 

r° 

sDzBjds -f- 

B=zds 

J-T 

J-T 


«T N 4f,Bi»r + % 

Finally, since yEX N (r) implies that y is a linear combination of {B^}, this equation 
can be rearranged to give the stated result. □ 


Our final lemma extends a standard convergence result for spline 
approximations to the situation here where we have a sequence of time-varying 
norms. 
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Lemma ^ Suppose {r M } is an arbitrary collection of functions in T and 


assume z(t) € SB for each t E [0,T]. Let us write P (r )i(r )z(t) = 


P2- M z 

P N,M Z (.) 


and 


z(t)- 


z o 

z() 


Then for each fixed tE[0,T] , 

|D(P N,M z — z) | m -» 0 as N -* oo 


and this convergence statement is independent of {r M }. 


Proof. 


I N,M t(r M )z(t) = 


Fix tE[0,T]. Using the notation of Lemma 1, we let 


N.M, 


z, (0) 

zrno 


Notice that zESB means that zEH l (-r,0) (and thus, 


^zEHH-t^O)). With a triangle inequality and use of the Schmidt Inequality 
(Theorem 1.5 in [19]), one can show that 

|D(P n ’ m z-z)I tM <; |D(P n ' m z-z^ m )I tM + |D(zr-z)l TM 

s k,(M.)i^ M z-zri TM + iKzr-z)i T „ 

<: k i(^)(l pN ' M z— zl T „ + |z-zT' m I tM ) + IO(z, N ' M — z)I tM 
where kj is a constant independent of N,t m or z, and r 0 is as in the definition of 
the set T. Now, 


|P N,M z — z| m £ It n,m z — z L £ ll N,M z — z|| M = |z" ,M — z| | M 

T T T T 

so that |D(P n,m z — z)I t m ^ 2k 1 [^]|z^' M — z|^ M + |D(z" ,m — z) I t „ . 

We can modifiy (2.16) of [19] slightly to show that for z6H‘[-t m ,0], 

|z — z^I^m <; k 2 [^j-)|D(z — z^ ,m )I tM £ k 2 (j^]|D(z — z^' m )I t m where k 2 is a constant 

which is independent of N,t m and z. Hence there is some constant C such that 

|D(P n - m z— z)I tM C|D(z?’ M — z)I t m . 

A slight modification of (2.17) of [19] yields |D(w - w"' M ) I tM £ (^)|D 2 w| tM 

^J|D 2 w| 0 whenever wEH 2 (-r,0), in which case it is clear that the stated result of 
the Lemma holds. We can now use the First Integral Relation for linear splines ([19], 
equation (2.10)) and the dense inclusion of H 2 (-r,0) in H^-r.O) to argue that the 
result holds whenever zEH^-rjO) and hence for any zE9B. □ 


14 



In order to prove our first theorem, we make an additional assumption about 
the unknown delay. We shall assume tET c s= *T D {t | (1 — t) ;>$ >0 a.e. tE[0,T]}. 
The assumption that the time derivative of the delay is bounded under and away 
from one is crucial to our convergence arguments. We would like to point out 
however, that this is not an unreasonably restrictive assumption in many modeling 
situations. Consider that this inequality is equivalent to constraining 
w(t) = t — r(t) to have positive derivative, implying that w is monotonically 
increasing. Since 0) gives the value of the lagged time, we can interpret this 
.condition as follows: Whenever one event precedes another ( wltj) < w(t 2 ) ), then the 
effect of the first event must be felt by the state before the effect of the second 
event (since the monotonicity of w implies tj < t 2 ). In both the models of [7] 
and [10], (when there is enough smoothness in the problem so that the delay is 
differentiable) this condition is satisfied. 

We now consider the case of time-dependent unknown delay. We define the 
approximation set for T as Tc = T c D S M . The discussion applied to T M in 
section 3 remains valid as applied to this set Tj?. Thus, the following statements 
are true: 

The set can be characterized as = / M T C . 

(HT C ) Given tET c , in C[0,T] as M-*oo. 

Each of Tc and T c is compact in C[0,T]. 


Theorem 1: Time-dependent Delay. Assume {p M } is an arbitrary sequence in 
lie = (S> ^c» ^)» with p M = (7 m , t m , 0 m ) and p M -* p 6 II C as M -* oo; this 
convergence statement means that 7 M -* 7 in E^XCJOJ], r M -» r in C[0,T], and 

-* 0 in RX C[-r, 0]. Then, using the notation z(t;p) = [ £ Z, where z is a 


solution of equation (3.1), and z N (t;p M ) 

solution of equation (3.2), it follows that 
each t E [0,T]. 


_ [ z> M ) 

[z N (-;P M ) 

|Zq(p M ) - 


| E X n (t m ), where z N 
z 0 (p)l -* 0 as N,M 


oo 


is a 
for 


Proof. We first observe that 
|z£(p M ) — z 0 (p)| ^ | t M z N (t;p M ) — i M z(t;p) |^ M 

£ || l M z N (t;/> M ) - P N ' M t M z(t;p) I/,- | P N * M i M z(t;/j) 
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We apply Lemma 1 to the second term above and conclude that it goes to zero as 
N, M -* oo, for each t£[0,T]. We now turn to the first term. Here we will use the 

notation 5* N,M z = T n (t m )z = 


pTV 

p N,M z 


♦ A straightforward calculation demonstrates 
that (all equations should be interpreted for a.e. t): 


(4.3) AIi M z N (t;4 M ) - P NM i M z<t;p) ^ - A|< M (z'W’> - T" M z(t;„)) j 


r M tt) 


= 2 • « z N (t;p M ) - ? NM z(t;p) , £[z N (t;„ M ) - T N ' M z(t;p)] » T „ (t) 

+ f M (t) • [z N (t;p M ) - P N - M z(t;p)T| . 

From now on, in order to minimize notation, we will omit the explicit time 
dependence and simply write, e.g., z (/>) instead of z(t;p). We use Lemma 2 and the 
fact that ^z N (p M ) — T N,M z(p)J E X N (r M ) to conclude that 

«^ N ' M zW , (z N (/j M )-g> N ’ M z(p)) »^M = , (z n (p m )-7 n ' m z(p)) »^m 


dt 


+ (^) f S [Dz(p)— DP N,M z(p)J [z N (p M ) - P N - M z(p)] ds 

T J -r M 

+ (^)f [z(p)— P N M z(/o)][z N (p M ) — P N,M z(p)]ds . 

T J tM 


Let us write this as 


«£? n - m z (p) , (z N (p M ) — 9 n,m z(/j)) » tM = «y N>M ^ , (z"{p M )-9 K ”z{p)) »^ M + E, 


where E 


dt 


1 = (JS] [ [s{Dz(p) — DP N,M z(p)} + {z(/j) — P NM z(/j)}][z N (yO M ) — P^zCalJds . 

J _ t m 


Then equation (4.3) is equivalent to 
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2 • «z N (p M ) - 5> N - M z(p) , 


dZ N (p M ) eb N,Mdz(/5) vx 

~ dt 7 1 TV 


- 2S, + r M [z N (p M ) - P N - M z(p)]1 . 

s*-T pq 

We now use the fact that [z N (p M ) — 9 N,M z(p)j € X n (t m ) C Z and equations (3.1) and 
(3.2) to rewrite the above equation as 


£||z N (p M ) - ? N - M z(p) ( t „ - 2 • «z N (p M ) - 3> N - M z(p) , T N - M ^(q M )z N (p M )» r M 

(4.4) - 2 • «z N (p M ) - ? N - M z(p) , ? N - M ^t(q)z(p)» T M + S 2 

= 2 • «z N (p M ) - ? N - M z(p) , T N,M £wA(q M )z N (p M ) - ^l(q)z(p))» T M + S 2 

where £ 2 = -2Ej -f- t M lz N (p M ) — P N,M z(p)l . Let us now consider 

J l 3»-T M 

<(z N (p M ) - !P N,M z(p) , 5> N ’ M (a(q M )z N (p M ) - ^(q)z(p))» r M 

= «z N (p M ) - 3> N ’ M z(p) , ^t(q M )z N (p M ) - ^l(q M )!P N ’ M z(p)» T M 
+ «z N (p M ) - ? N ’ M z(p) , ^(q M )? N - M z(p) - ^l(q)z(p)» TM , 
which we will define as Ti + T 2 . We can estimate T t as follows: 

T t = «z N (p M ) - ? N - M z(p) , ^(q M )z N (p M ) - ^t(q M ) c P N ' M z(p)» T M 

= [z :”(p M ) - P^- M z(p)][F(t,q M ,zS , (p M ),z N (p M )) - F(t,q M ,P”- M z(p),P N>M z(p))] 

+ [ [z N (p M ) - P n - m z(p)][d(z n (p m ) - P N,M z(p))]ds 
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<: |z”(p M ) - P^z(p)\ | F(t,q M ,zg(p M ),z M (/> M )) - F(t,q M ,P^ M z(p),P N ' M z(p))| 

+ | [ d[(z n (p m ) - P N>M z(/5)) 2 ]ds 

J _jM 

<; |z”(p M ) - P^- M z(p)| {m(t)(|z> M ) - P£- M z(p)| + |z N (p M ) - P N - M z(p)I t m 
+ |(z N (p M ) - P N M z(p))| s _ t J)} + |(z N (p M ) - P NiM z(p)) 2 [ 

S “ S»-T^ 

= m(t)|z£(p M ) - P"- M z(p)\ 2 + m(t) | Z q(p m ) - P^ lM z(p)| |z N (p M ) - P n,M z(p)| t m 
+ ^ |z > M ) - P^- M z(p)||^ |(z N (p M ) - P NM z(p))| ^jj 

+ | (z”(p M ) - Po ,M z(p)) 2 - | (z N (p M ) - P HM z(p)) 2 | m , 

where we have used assumption (HF) in the last inequality and c is any positive 
constant. Thus, 

T t <: (in(t)+i)|zSCo M > — 

+ S^lzJOA - PTzOOl 2 + |I*V) - P N ' M zWI 2 „ 

+ Bg)|zS<„ M ) - Pj' M zM| 2 + |(z N (o M ) - ^zwfl 
- !(« - P N ' M zW)1 . 

s»-T m 

The term T 2 satisfies: 

T 2 - «zV) - f NM z(s) , U(q M )T N ’ M z(rt - U(<i)z(o)» tM 

£ ||z% M ) - T N - M z(rt^ M + S|u(, M W"- M zO>) - Mi)z(p)( 7M . 
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But I ^4.(q M )7 N - M z(p) — .4(q)z(p)||^ M = | F[t,q M ,Po' M z(p),P N,M z(p)) — F[t,q,z 0 (p),z(p)]| 2 


+ |D(p N>M z(p) - z(p))| 2 M 

so that we have 

T 2 £ |flz N (p M ) — T N - M z(p)|£ M + ||F(t,q M ,Po’ M z(p),P N ’ M z(p)) — F{t,q,z 0 (p),z(p)] | 2 
+ | |D(p N,M z(p) — z(p))|^ M 

We now combine the above estimates with equation (4.4) to obtain 

(4.5) £||z N (p M ) - 5> N - M z(p)|£ M £ (2m(t) + 1 + (1+ i)m 2 (t)] |z"(p M ) - P”- M z(p)| 2 

+ |z N (p M ) - 9> N - M z(p)| 2 M + (c - 1) (z N (p M ) - P N - M z(p))1 

T 1 } U-r M 

+ jz N (p M ) — T N,M z(p) + |F[t,q M ,Po’ M z(p),P N ' M z(p)) — F(t,q,z 0 (p),z(p))| 2 
+ |D(p N>M z(p) - z(p))| 2 M + t M [z N (p M ) - P N ’ M z(p)] 2 | 3 - 2E, . 

We can obtain a bound on Sj as follows: 

2-ISJ ^ 2- |(i^)| f |s {Dz(p) - DP n ’ m z(p)> + {z(p) - P N,M z(p)}| |z N (p M ) - P N ' M z(p)| ds 

T J_ t m 

£ |t M |[|Dz(p)-DP N ’ M z(p)| 2 M + |z N (p M )-P N ’ M z(p)| 2 M ] 

+ |^|[|z(p)-P N ’ M z(p)|; M + |z N (p M ) — P N,M z(p)| 2 M ] . 
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If we define W c (t) = |2m(t) + 2 + d + g)m 2 (t) + /z[l + ^)l and 

h N ' M (t) - (1 + Ai) \Dz(p) — DP N,M z(/?)|^. M + I F^t,q M ,Po’ M z(/7),P N ’ M z(p)] - F(t,q,z 0 (p),z(p))| 2 
4- yi IzO>)— P n,m z(p)|^ m 
then we have a bound of the form 

(4.6) £||z N (p M ) - T N > M z(/>)f TM £ W c (t)lz N (p M ) - ? N -”z(p ) ( rM + h N - M (t) 

+ (c - 1 + T M (t)|zV) - P N,M z(p)) 2 | . 

By assumption, t m €T c for all M, which means we can choose c = | and we are 
assured that — 1 -f* T M (t)J^0 . Let w=w c for this value of c. Then, equation 

(4.6) implies 


£||z N (p M ) - T n - m z(p)|| 2 m £ w(t)||z N (/> M ) - T N - M z(p)f TM + h N - M (t) . 


We can now use the Gronwall Lemma to see that 


|z N (/> M ) - T N,M z(/o)|| 2 M (t) <; fz N (p M ) - T N,M z(p) ||^ m ( 0) Je"^° W(9)d9 ) + [ e J 

J n 




+ [V»““V' M (s)ds 

J n 


^ |0" — 0 L e ' u + | e “ h (s)ds for every tG[0,T], 

where we have used the following estimate for the initial data: 


|zV> - T N-M z(p)I^ M (0) - | t P"(r M (0))z(0ip M ) - c P' j (t m (0))z(0;p) £ 


to) 


Our parameter convergence assumption and our assumption (HF) guarantee that 
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1 ^ ^ tj2 pT 

— II -»0 as M-«<» and [ w(s)ds<oo. Therefore, the final step is to argue that 
n o Jo 

f T h N,M (s)ds -» 0 as N,M -* co. 

J o 

To this end, consider that 
|F(t,q M ,P£’ M z(p),P N ’ M z(p)] — F(t,q,z 0 (/>),z(/)))| ^ 

I F(t,q M ,Po ,m z( / o),P n,m z( / £>)) — F(t,q M ,z 0 (/»),z(/o)) | + | F(t,q M ,z 0 (p),z(p)) — F[t,q,z 0 (/>),z(p)] | 

^ m(t) 1 1 Po’ M z(p) — z 0 (p) | + |P N,M z(p) — z(p)I t m + |(p N,M z(/o) — z(/o)]| J 

+ |F(t,q M ,z 0 (/o),z(/>)) — F[t,q,z 0 (p),z(p))| . 


Thus we can argue that 

h N,M (t) £ CM[\Dz<.p)-DP"- M z(ptf T „ + ly N ’ M z(p)-z(p) 

+ | F(t,q M ,z 0 (/?),z(p)] — F[t,q,z 0 (/o),z(/))) | 2 ) 

where C(t)GL 2 (0,T) is independent of N and M (it does depend on m(t), r 0 , p.) . 
Finally, Lemmas 1 and 3, the continuity of F in q (this follows from the 
assumptions about f and the fact that z(t;p)E3B implies that z(t; p) 6 C(-r, 0) ), and 
the convergence q M -*q (following from p M -*p) imply that h N,M (t)-»0 as N,M-*oo 
pointwise in t; the dominated convergence theorem can now be applied to deduce 

that f T h N,M (s)ds -* 0 as N,M-»oo, and this concludes the proof of the theorem. □ 

*o 

Theorem 2. Let {p N,M > with p N,M - (7 N,M , t n,m , £ n - m ) 6 Q X X 3 C Q X T c X 3, 
be a sequence of solutions to {(P N,M )}. Then, given (HG), (HT C ), and (HI), there exist 
p* = (T*»t*, 0*) E QXT C X3 and a subsequence {p N|< ’ Mk } such that p k ' ' i -*p* with 
Nk> (the meaning of this convergence statement is the same as that of 

Theorem 1), and p* is a solution of (P). 
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Proof. The existence of the convergent subsequence with limit p* 


in 


QXT C X5 follows from the compactness of the parameter sets. We would like to 
show that this limit is a solution of (P), i.e., that Kp*)<,i{p) for any p€QXT c X3. 
Consider any element, p , of QXT C XJ. By definition, J N|< (p Nk ' M| ‘) ^ J N|< (p) for any 
pGQXTcXJ, and, in particular, J Nk (p N|<,Mk ) ^ J Nk (/ Mk p), where we are using the 


shorthand notation / M|t J = (7,/ Mk r,J). Let N^, Using the second statement 


of (HT C ), we can conclude that I p-*p. Thus, we can apply Theorem 1 to conclude 
that J(p*)^J(j5). This proves the theorem. □ 


For our second class of problems, we consider equations of the form (2.1), 

i* 

with the delay determined by g(x(s))ds = 0, where we shall assume 

_ ■* t-r(t) 

(g,0) belongs to the set T g . By differentiating, we can write this in the equivalent 
form 


r(t) = 1 


g(x(t)) 

g(x(t-r(t))) * 


T(0) = Tj 


where Tj is determined by g(x 0 (s))ds = 0 (we assume that x 0 and 0 

J- T i 

are such that r- exists). This (the integral form for the delay) is the type of 
model equation considered in [10] and [11] (see especially [11] for questions of 
existence and uniqueness of solutions). The model equations of [7] are of a similar 
nature; a system of delay equations with multiple delays is considered, where each 
delay satisfies an equation of the form f(t) = G(x(t),x(t-r(t))), where x represents 
a vector state (eg., see equation (4) of [7]). We do not address these equations 
directly, but note that the ideas developed here should extend, as the delays of the 
type in [7] also satisfy the conditions required for our methods. 


With the differentiated form, it is more convenient to consider that Tj is 
unknown (rather than 0), so now we shall assume that we estimate and recover 

0 from the above relationship. It is readily seen (using both of the equivalent 
integral and differential forms for r) that if (g,0)GTg, and x and x 0 are any 
continuous functions, then r and Tj satisfy: 

L £ Tj £ T 

r <; T(t) <[ t, |r L <; M and (l — i(t)] ^ S for all t£[0,T] 
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where T = 0/g u , t s= 0/g L , S = g u /g u , and // = |l — $ _1 |; i.e., Tj belongs to a 
compact subset of R, and tET c . Using the notation of section 3, we write the 
above differential equation for r as 


r(t) = 1 - 


g(z 0 (t)) 


! T<0) Ti ' 

In the parameter estimation problem, we shall assume that g and Tj are unknown, 
and represent our unknown delay terms by the pair g ss (g, Tj). We shall still use 
the notation gETg (the compact set for 0 being replaced by the corresponding 
compact set for Tj). We must approximate the function g, as discussed in section 
3. We are thus led to the consideration of a corresponding approximating equation 
for the delay: 


■ n.m 


(t) = 1 


g M (z”(t)) 


g M (z N (t)|_ T N,M (t) )J’ 


T ,M (0) = Tj 1 , 


where g M = (g M , Tj) E Tg (the notation was established in section 3). 
Unfortunately, however, the sequence (t n,m ) may not satisfy the constraints of 


T c , as there is no guarantee that T 
therefore define 

N.M, 


N.M 


remains positive for all tG[0,T]. We 


X N,M (t) = 


(t) 


max (21> T N,M (t)] 

where T is defined above, and 


(4.7) 


t N ’ M (t) = 1- 


J M (zg(t)) 


g M (z N (t)| nmJ 


X N,M (t); 


t n,m (0) 


t-M 


-T -W 

Our approximation system, then, is the coupled pair of equations (3.2) (solved with 
q M = (7 m ,t N,M ) ) and (4.7). That a solution of such a system exists (and is unique) 
is more easily seen in section 5, and we thus postpone that discussion, assuming for 
now that this is the case. It can be argued that t n,m a solution of (4.7) belongs to 
T c for all N,M. It is clear that all t n,m satisfy (1 — f N,M ) ^ J ( S defined above) 
and it is not difficult to show that, for all tG[0,T], and all N,M 

— ~ l^ -1 — 1 |T/t n.m... _ l/r 

re “ (t)^Te‘ 


In this case then, the above will define the numbers T 0 and r used in the 
definition of T. The functions {\ N,M } enjoy certain properties as well. For later 
reference, we collect the various results in the following lemma. 
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Lemma 4. Let {x N } and {y N } be any functions satisfying x N 6 C[0,T], 
y N 6C([0,T]X[-r,0]) l let X N,M be defined as above, and let r N,M be the solution 
of 


. N,M 


(t) 


g M (x N (t)) 
g M (y N (t, r N,M (t))) 


X N,M (t); 


T (0) = T 


M 


Then for all N,M : 
t N,M G T c , 

I X N,M L ^ 1, and 

|l — X N,M | <; ^ |t — r N,M | for each tG[0,T], and any tGT c . 


We are now ready to prove a convergence result for this class of state-dependent 
delays. 


Theorem 3: State-dependent Delay. Type L Assume {p M } is an arbitrary 

sequence in II M s (Q, Tg, 3), with p M = (7 M , g M , 0 M ), and p M -* p G II = (Q,T g , 5) as 
M -► oo; this convergence statement means that 7 M -» 7 in R^ X C[0,T], g M -* g 
in C[x,x], r-' 1 -» Tj in R, and $ M -»0 in R X C[-r,0]. Then (using the same 

notation as that of Theorem 1), it follows that |t n,m — r| + | Zq(p m ) — z 0 (p) | -* 0 
as N,M -* oo for each t G [0,T]. 

Proof. We begin as with the proof of Theorem 1, arriving at the following 
analogue of equation (4.3): 

(4.8) ^|i M z N (t;p M ) - P N>M i M z(t;p) + £|r N ' M (t) - r(t)| 2 

= 2 • «z N (t;p M ) — T N - M z(t;p) , ^[z N (t;p M ) — ? N,M z(t;p)] » T N.M (t) 

+ t KM (t) • [z N (t;/> M ) - P N - M z(t;p)]1 N M + 2 • (r N ' M - r) (t N M - t). 

s=-T ’ (t) 

Our analysis may now proceed exactly as in the proof of Theorem 1 up to equation 
(4.5), except we shall replace by = Sj + (t n,m — r) (f N,M — r). With this 

modification, the analogue of (4.5) is: 
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(4.9) £(||z V) - T n ' m 2(p)S^ n .m + k N ’ M - r| 2 ) 

£ (2m(t) + 1 + (1 + |)m 2 (t)) |z”(p M ) - P£- M z(p)| 2 
+ |z N (p M ) - T N,M z(p) | 2 n.m + (c - 1) (z N (p M ) - P N - M z(p)]1 

T ^ J | _N,M 

S*-T 

+ |z N (p M ) — ? N,M z(p)||* NM + !F[t,q M ,Po ,M z(p),P N,M z(p)] — F[t,q,z 0 (p),z(p))| 2 

+ |D(P N ’ M z(p) — z(p))| 2 NM + t n,m [z N (p M ) — P^zCp)] 2 ) N — 2 S 1 . 

Now, in order to bound £ lf we first consider the new delay term; using a triangle 
inequality and the statements of Lemma 4, we obtain: 

(T - _ - « - (/" - r» {(• - )X- - (1 - «)} 

, |N.M lUN.Mir g M (z£(p M )) ) fi g(z 0 (p)) 1 
* |t “ t||x 

+ |t nm - t| (\ nm - 1) r 1 - . 

e[z(p)|_ T J 


Thus, 


(4.10) 


, N.M . , • N.M • v 

(t — t)(t — r) 


. I N.M _ I g(z 0 (p)) _ g M (Zp(p M )) 1 1 — ^~M | N.M _ I 2 

U*»tr) «1^ M )|. t km) ~ T 


*=» CTj + cr 2 . 


The term < j x satisfies 


r < |t N,M t 11 g M (z 0 (p)) g M (z 0 (p)) g M (Zp(p )) 

1 l|g(z(p)L T ) g(z(p)|_ T ) g(z(p)|_ T ) g(z(p)|_ T ) 

g M ( Z p(p M )) g M (z”(p M )) | 

g(z(p)|_ T j e M (z N (p M )|_ T N, M ) J 
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s \ k N,M - t | 2 4- \\ &(z 0 (/>)) - g M (z o(/>)) f + k|| Z 0 (p) - z^ M )| 2 

8 l 

+ ^k N,M - t| g M (z N (p M )| N , M ) - g(z(p)L T )| 

6 l I 

£ \ |r N,M — t| + 5 1 gfeo(p)) — i M (zo(p)) f + A*g | z 0 (p) — Pq M z(p)| 

Bl ^ 

+ m||Po’ M z(/>) — ZoO> M )| + |j|T N,M — T l|^g|z N (p M )| t n,m — z(/ j )|_ t n,m| 

+ A«g| z (/ > )|_ T N,M - Z (/>)L T | + |g M (z(/>)|_ T ) - g( z (p)|_ T ) |j 
^ h + sfe^g) 2 +^y 2 k N - M -T | 2 + //||Po' M z(p)-z> M )f 

&L &L , 

+ ^i|_|r N,M — t|[ z N (p M )| NjM — P N,M z(/>)| N>M +1 P N,M z(p)| n,m z(p)| NiM 

g L [ -T -T I -t -r 

+ p.\\z 0 {p) — Pq' m z(p)| + i|g(z 0 (/j)) — g M (z 0 (p)) f + ljz(p)| ^ NiM — z(/o)|_ T j 2 
+ ||g M (zCp)| T ) - g(z(p)|_ T )| . 

Since we may identify z(t \p) with (x(t), x^( • )) T , which is a solution of the original 
delay equation (with the parameter p), we see that 

j z(p)\ ^ N>M — z(/?)|_ T j £ | Dz(p) |Jt- t n,m | £ ^ T _ ^ M ^ a " d theref ore wo 

can write (for some constants K lt K 2 which depend on the various constants, g u , g L , 
H g, etc., but not on N,M ): 

<; kJt n,m - t| 2 + Aig|Po’ M z(p) - Zq(p m )| 2 

+ K 2 |t n,m — r| z N (p M )| n,m — P^zO?)! N ,M 

-T -T 

+ i| P NM z(p)^ NM — z(p)| ^ NM f + /z||z 0 (p) -Pf^f 
+ ^|g(z 0 (p)) — g M (z 0 (p)) f + ^|g M (zCp)|_ T ) — g(z(p)|_ T )| 
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and finally, for some positive constant c (to be determined later) we have 


o-i ^ (Kj + ^?)|t n,m — t| -f- | z n (p m )|_ tNiM — P N,M z(p)|_ t n.m 

+ //g|Po' M z(/5) — Zo(p M )| + i P N,M z(p)| ^ NjM — z(/?)| ^ N>M 

+ »g\z 0 (p) -Po’ M z(p)f + \\ g(z 0 (p)) - g M (z o 0>)) | 2 

+ ^h M M. T ) - eM. T )| 2 . 

Combining the above estimate with equation (4.10) and the estimate for Ej obtained 
in the proof of Theorem 1, we have now shown that 

2 \±,\ <: //(|Dz(p)-DP n ’ m z(p)|^ N jM + |z N (p M ) — P N,M z(p)|^ NM ) 

+ |^jlz(/o) — P N ' M z(p)|^. NM + |z N (p M ) — P N,M z(p)|* NM J + kJt n,m — rf 
+ c|(z%o M ) - P N,M z(/>))| n m V 2^||Pq' m z(/j) - z> M )f 
+ (p NM z(p) — z(p))| NM + 2 ^||z 0 (/7) — Pq ,M z(/7)| 

-T ’ 

+ |g(z 0 (p)) - g M (z 0 (p)) f+ |g M (z(p)|. r ) - g(z(p)|. T )| 2 . 

Proceeding as in Theorem 1, we define 

to- = |2m(t) + 2 + (1 + |) m 2 (t) + + 4* 2//g + KJ 

and 

h NM (t) = (l + /i)|Dz(p) — DP N,M z(p)|^ NM + | (p N,M z(p) — z(/>)]| nm 

+ | F[t,q M ,P^ M z(p),P N M z(p)] — F[t,q,z 0 (p),z(p)] j + ^|z(p) — P N,M z(p)|^ N M 
• + 2At||z 0 (p) — Pq’ M z(p)| + | g(z 0 (p)) — g M (z 0 (p)) f+ |g M (z(P)|_ T ) — e( z M_ T )f 

and then equation (4.9) becomes 
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d_ 

dt 


zW - tP NM z(p)l’ N . M + |r N ' M - rf) 

S W- (|2'V") - ‘P"' m z(/>)| 2 nm + w - rf ) 

+ (c - 1 + t N ' M + c) [W - P ,tf Wf| nm + h""(t). 

3 *-T 


Because each r N,M belongs to T c , we are assured of being able to choose c and 


c such that (c — 1 + t N,M + c) <. 0 for all N,M. Thus, we can write 

.(4.11) i(|z"Cp M ) - + |r MM - rf ) 

£ v (|*V) - » w z<,»£ w + It"-" - rf ) + 

An application of the Gronwall Lemma now gives: 
fz N (p M ) - T N - M z(p)i^ NM (t) + |r N ' M - rf(t) 

JO 

for each t€[0,T]. Given that 0 M -» 4 > and Tj 1 -♦ Tj, our final step is to argue that 
h N,M -♦ 0 in L x (0,T). The function h N,M is composed of the same terms as in the 
time-dependent delay case, with some additional terms here. We can begin in the 
same way as in the proof of Theorem 1 to bound h N,M , additionally considering the 
new terms. Thus we can derive 

h N ’ M (t) <: C(t) (|Dz(p) - DP N,M z(/?)|^ N M + [ ? N ' M z(p) - zip) |^ NM ) 

+ I F[t,q M ,z 0 (p),z(p)) — Fft.q.ZotpJ.zt/o)) | 2 

+ | g(z 0 (p)) — g M (z 0 (p)) f + |g M (z(p>L T ) — g(z(p)L T )| • 

The second term above satisfies 

|F(t,q M ,z 0 (p),z(p)) — F(t,q,z o 0>),z(/>)]| = 

|f[t,7 M ,z o 0>),z(p)|^ ^ NM ,z(p)] — flt.TjZoC^zC/oJ^ ^zCp)] 

^ |f[t,7 M ,z o 0>),z(p)| s _ t n jM ,z(p)] — f [ t, 7 m , z 0 (p), z(/ 9)| s- _ t , z(p) ] j 

+ |f[t,7 M ,z 0 (p)»z(p)| s __ T .z(p)] — f[t,7»z 0 (p),z(p)| s __ T ,z(p)]| 
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^ m(t) z(p)| N>M — z(p)| _ 

I S--T S T 

+ |f[t,7 M ,z 0 (p),z(/[?)| s= _ T> z(p)] — f[t,7»z 0 (p),z(p)| s __ T ,z(p)]|. 

^ m(t) (-rS^T^ (t) ^ TNM ~ T l 

+ |nt,7 M ,z 0 (p),z(p)|_ T ,z(p)] — f[t,7,Zo(p),z(/7)|_ T ,z(/>)]|. 

^ C(t) |r N ’ M - r| + |f [ t, 7 m , z 0 (p), z(p)|_ T , z(/?) ] - f[t,7,z o 0>),z( / 4 T ,z(p)]|. 

Finally, we shall redefine W = CJ + 2C 2 in equation (4.10) (thereby removing the 
first term above from the definition of h NM ), and then we write 

h N,M (t) ^ C(t)[|Dz(p) — + I T N,M z(p) — z(p) | tNM ) 

+ 2 1 f [ t, 7 m . z 0 (p), z(/?)|_ T , zip) ] — f[t,7,z 0 (p),z(/i7)|_ T ,z(/5)]| 2 

+ | g (z 0 (/j)) — g M (z 0 (/>)) f + |g M (z(p)|_ T ) — g(z(/7)|_ T ]| . 

We can now see that the desired convergence follows from Lemmas 1 and 3, the 
continuity of f and the convergence 7 M -* 7i and the convergence g M -♦ g. This 
concludes the proof of the theorem. □ 

We can also prove an analogue of Theorem 2 for this state-dependent case. 
The proof follows almost word for word; we must only replace the statements of 
(HT C ) by the corresponding statements for the set Tg (these properties have been 
discussed in section 3, we compile them here for reference): 

The set Tg can be characterized as J M Tg. 

(HTg) Given g£T g , J^g-^g in C[x,x] as M-*co. 

Each of Tg and Tg is compact in C[x,x]; each of Tg and Tg is 
compact in C[x,x]XR. 

Theorem 4. Let (p N,M > with p N,M = (7 N,M , g N ‘ M , £ N,M ) eQXT£X3CQXTgX5, 
be a sequence of solutions to {(P N,M )}. Then, given (HG), (HTg), and (HI), there exist 
P* = (7*. g*» 3*) £ Q X Tg X 3 and a subsequence {p Nk ’ Mk } such that p Nk ’ M|< -♦ p* with 
Nk» (the meaning of this convergence statement is the same as that of 

Theorem 3), and p* is a solution of (P). 
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For our final case, we consider equation (2.1) with a delay of the form 

T(t) = g(x(t)), where the function g is assumed unknown and is to be estimated. 

Using our abstract notation, this is written as r(t) = g(z 0 (t)). In order to prove a 
convergence result, we reformulate our delay in terms of a differential equation. 
Thus, as long as we consider geW 1 '*(x,x) and z 0 € W l, “(0,T) the unknown delay 
satisfies an equation of the form 

t = Dg(z 0 (t)) z 0 (t) ; t( 0) = g(0(O)), 

which is equivalently written as 

t = Dg(z 0 (t)) f (t, 7> z 0 (t), z(t)|_ T(t) , z(t)] ; t( 0) - g(*>(0)). 

For this form of the delay, we must make additional assumptions both about f and 

the unknown function g in order to prove a convergence theorem. 

(HF) There exists a constant 7 such that for any 7GQ, and any 
(£, C » V) £ R X R X L 2 (-r, 0), it f ollows that | f (t, 7, £ , f , 77) | <; «F. 

Let (z,r)g represent a solution pair of the above equation with a particular g£T g , 
coupled with equation (3.1) for the state. 

(HTg) Assume we search for g in the set D { g G W 2, °°(x, x) | |D 2 g|^ /i g 

and (z,r)g satisfies r(0 = goz 0 GT c }. 

It is no longer clear in this case of state-dependent delays how to restrict the set 
of unknown functions in order to guarantee that tGT c . This particular 
formulation of delay is perhaps not so natural. We note that the previous 

formulation was motivated by specific scientific phenomena ([7], [10]), whereas the 
present formulation comes from a more mathematical perspective (eg., our example 
problem in the final section comes from [16], where no underlying model has been 
considered). Nonetheless, it is still possible to obtain some results for this case. 

Let us assume we begin with some compact (in W 1, “) subset of T g (without 
changing notation - i.e., now let us consider that T g is compact), and define our 
approximation subset as = l M ^gi this set T g is also compact, however the 
compactness is no longer easy to characterize, as was the case for the previous 
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types of delays. We could begin, as before, by considering the analogue of the 
above equation with g replaced by g M and z 0 replaced by Zq, however, we 
could not then guarantee that the resultant r N,M belongs to T c . Thus we will 
instead consider the following equation to determine the approximating delays: 

x N,M (t) = maxfCl — 5), Dg M (zg(t)) • f ( t, 7 M , z£(t), z N (t)| NM ,z N (t)) • X N,M (t) ) 
(4.12) ' T ' (t) 

T N ’ M (0) = g M (Zg(0)), 

where X N,M is defined in the same way as in the previous class of delays. Our 
approximating equations are then the coupled system (3.2) and (4.12), and again it 
can be argued (assuming for now that the system has a unique solution) that r N,M 
a solution of (4.12), and X N,M will satisfy the conclusions of Lemma 4. 

Theorem 5: State-dependent Delay. Type IL Assume (/? M ) is an arbitrary 

sequence in II M s (Q,Tg,3), with /> M = (7 M , g M , 0 M ), and -* p G II = (Q,T g ,3) as 
M -» oo; this convergence statement means that 7 M -* 7 in R. u X C[0,T], g M -* g 
in C[x, x], Dg M -* Dg in L“[x,x], and in R X C[-r,0]. Then (using the same 

notation as that of Theorem 1), it follows that |t n,m — r| + | z£(p M ) — z 0 (p) I -» 0 
as N,M -» co for each t G [0,T]. 

Proof. The proof follows the same outline as those of Theorems 1 and 3, so 
we shall only present the differences. Consider first that 

(r NM — r)(r N,M — t) ^ |r N,M — r| {|f(t,7,z 0 ,zL T ,z)||Dg(z 0 ) — Dg M (Zg)| 

+ |Dg M (Zo)||f(t,7,z 0 ,z|. T ,z] — f(t,7 M ,Zo,z N |_ T N,M,z N ) • X N M |} 

^ |r N,M — t| |Dg(z 0 ) — Dg(Zg)| + <5 |r N,M — r| |Dg(zg) — Dg M (zg)| 

+ k|t n - m -t| 

+ |f(t, 7, z 0 , z)|_ T ,z] - f(t,7 M ,z^,z N |_ TN .M,z N )|} 

where we have used the properties of X N,M and (HF), and K is a constant which 
does not depend on N,M. It then follows (using the fact that gGTg and (HF)) 
that 


31 



(T N,M - r)(t^ - T) £ 


N.M 


(^! + f + 9 + y)| T w - T f + 1 ]zo - 

+1 |D S (zg) - De M (zS>r + 3 |f{t,7,z„zL T ,z) - f(t,7 M ,z 0 ,z|_ T ,z)| 2 
+ K |r N,M — r| m(t){ |z 0 — z£| + |zL T — z n |_ t n,m| + |z — z n | t n.m} 


N,M 


£ w x (t) (|r NM - t| + lz N (/) M ) - 7 n ‘ m z(p) 1 tNiM ) + c x ||p N ’ M z - z! tNJ 
+ i |Dg(z£) — Dg M (z£)| 2 +i |f(t, 7» z 0 »z|_ T , z) - f(t,7 M ,z 0 ,zL T ,z)| 2 
+ m(t) K |r N,M — r| |z|_ T — z n |_ t n, m | 

where W 1 €L 2 [0,T], CiGlR do not depend on N,M. With several triangle 
inequalities, and letting c be some positive constant to be determined later, we see 
that 

(r N ' M _ t)( t n ' m - t) <; 

Wi(t) [|r N,M — r| + ||z N (p M ) — ? n - m Hp)||* n>m ) + C x ||p N,M z — zl^ NM 
+ ^ |Dg(zg) — Dg M (z^)f |f(t,7, z 0 , z|_ T ,z) — f(t,7 M > z 0 , z^z]^ 

+ m 2 K 2 |t n,m — t| + 1 |zL r — z|_ t n , m | 2 + \ |(z — P n,m z)|_ tN>m | 

+ < f |r N ’ M - rf + | |(P N ’ M z - z n )|_ tN>m | 2 . 

We can bound the term |z|_ T — z|^ NM | 2 exactly as in the arguments of Theorem 3, 
so that we shall finally write 


( T N ’ M _ T )(jr'^' _ f) £ 


N,M 


Orffl (It n ' m - rf + |zV> - ? w zW|f TN . M ) + C, |p w z - zf T „. 
+ j |£>etzg) — Dg M (zJ)| 2 +1 |f(t,7,z 0 ,zL T ,zj — r(t,7 M iZotZ|. T ,z]| 


,N,M 
|2 


+ \ 


(z - P N,M z)| 


+ \ (P N,M z 




We can now follow the proof of Theorem 3: We have 
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2|£ t | <; fi |Dz(p) — DP N ‘ M z(p)|^_ N>M + |z N (p M ) - P N ’ M z(p)|^. N>M 
+ ^ jjz(/>) — P N,M z(p)| 2 N M + |z N (p M ) — P N,M z(p)| 2 NM j 
+ 2w 1 (t) (|r NM - rf + ||z N (p M ) - ? N ' M z(p)£ N>M ) + 2C X ||p n ' m z - z^ NM 
+ |Dg(zp) — Dg M (Zo)| 2 + |f(t, 7> z 0 , z|_ t , z) - f(t,7 M ,z 0 ,z|. T ,z)| 

+ |(z - P n,m z)|_ tNiM | + c |(P N,M z - z N )| rN>M | . 

We define Wg = |2m(t) -f- 2 -f- (1 -j- g)m 2 (t) -f- -f- 2u>j(t)| end 

h N,M (t) = (l+/z)|Dz(p) — DP N,M z(p)|* NM + | (p N,M z(p) — z(p)]| nm 
+ | F(t,q M ,Po ,M z(p),P N ’ M z(p)] — F[t,q,z 0 (p),z(p)] | + ^|z(p) — P N,M z(p)|^ NM 
+ |f(t,7, z 0 ,z|_ T , z) — f(t, 7 m , z 0 ,z|_ t ,z)| + |Dg(zg) — Dg M (z^)| 2 
+ 2C Jp N ' M z - zfl* N>M , 

and, choosing our constants appropriately, we conclude that 
£(h%> M > - + |t NM - rf ) 

Z w (IzV) - ? NM z(„)||^ NM + |t nm - rf ) + h NM (t). 


The Gronwall Lemma now gives 


Jz N (p M ) - T N - M z(p)l rN(M (t) + |r NM - T I (t) 

s E + |g M (Pr# M ) - eW0))f ) ° ! ° aM ‘ + fV» T “ , ’ a V- M ( s )d S . 

J ° 

We can use the fact that g M ,gGTg and our assumptions that the parameters 
converge to show that 


| g M(pN.M£ M) _ g(0(o)) | ^ ki |£ _ ^ + kj |pN.M^ _ £|^ nm + |gM ( 0 M (o)) _ g( 0 M (())) | 

and now the statement of the theorem follows from the argument that h N,M -* 0 as 
N,M -» oo in L t (0,T) (this argument is now essentially the same as that for the 
proof of Theorem 3) and the assumed convergence of the parameters. □ 
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Theorem 6. Let {p NM } with p N,M = (7 N,M ,g N,M ,^ N - M ) € QXT^XJ C QXTgX J, 
be a sequence of solutions to {(P N,M )}. Then, given (HG), (HT g ), (HI), and (HF), there 
exist p* = (7*»g*»0*) £ QXTgXJ and a subsequence {p Nk,Mk } such that p Nk ' Mk -»p* 
with N^, Mjj-^oo (the meaning of this convergence statement is the same as that of 
Theorem 5), and p* is a solution of (P). 

Proof. As in the proofs of Theorems 2 and 4, the existence of the 
convergent subsequence is a consequence of compactness. That the limit p* is a 
solution of (P) can be argued as follows. By definition of Tg we can write 

g N ’ M __/ M g N,M g N,M g Tg. The compactness of Tg implies there exists a 

subsequence g with g -*g ET g (this convergence is in the W 

topology). We can now see that 



where | • |j denotes the norm in W 1 ’* 0 . Because all g€Tg have a uniform bound 
on |z> 2 g |^, we can use equation (2.21) of [19] and the fact that g Nk ’ Mk g * to 
conclude that g Nk ’ Mk g*. By definition J Nk (p Nk ’ M|< ) <; j Nk (j M|c p) for any pell, 

where we are using the notation / k p = (7, / Mk g, 0). We now obtain a proof of this 
theorem by letting and -+ oo, and applying Theorem 5. 


5. Numerical Implementation. The abstract equation (3.2) is equivalent to a 
system of ordinary differential equations, as we shall now demonstrate. Consider a 
fixed parameter pE II (actually, II M ). As the resulting system of differential 
equations is very similar in form, whether we are considering the time- or state- 
dependent delay case, let us write r(t) with the understanding that if the delay is 
state-dependent, this represents the solution of the appropriate differential equation 
involving the state and the unknown parameter(s). At any time t(=[0,T], z N (t) 
belongs to the (N+l)-dimensional space X N (r), which is equivalent (numerically) to 
X N (r); in the following, we will write z N , but it should be understood that we mean 


t(r)z N . Given that X N (T) = span{Bj}, we can write == B|Cs; T (t)) 

1 i-0 


w 0 (t) 

N 

EWjBj 
i»o i * 


34 



w 0 (t) 

and z N = ; here we have used the properties of the splines that 

SjwA + Wi^) 

B o (0)= 1 and B^Ol-O for i^O, and ^^ = 0. Since z N (t), 2 N (t)€X N , then 
P N z N (t) = z N (t) and P N z N (t) = z N (t). Thus equation (3.2) is equivalent to 

«2 N («,B i » T(tJ = «U N (t;q)z N (t),Bi» T(t) 

for i=0,l,. ..,N 

« z N (0) , Bj » T(0) - «3,Bi» T[0) 
and, using the representations above, we can write: 

Q(t)w(t) = H(t) w(t) + T(t)w(t) + F(t,w) 

(5.1) 

Q(0) w(0) = C 


where w = (w 0 ,Wi,..., w n ) t , Q, H, and T are (N + 1) X (N + 1) matrices with entries 

pO pO 

Q ifJ -«%» S j» T “ BiWJB/O) + Bj Bjds, Hjj = < Bj , DB j ) T - I BjDBjds, 

’ " J -T(t) J -T(t) 


3Bi f° 3B : 

and Ty = — (B^-g^)^. = — Bj-g^ds, for i,j = 0,1,.. .,N, and F and C are 

J -T(t) 

vectors in R Ntl with F(t,w) = [F(t,q,w 0 (t),2w k (t)B k ), 0, ..., o]^ and Cj = , Bj » T(0) . 

It may appear at first that as we iterate on the unknown parameters, we must 
recompute the entries of our matrices each time; in fact, this is not the case. Let 
us define the ’’reference” basis functions {B^} by B^(x) = while each B^ is 

defined on the interval [-r(t),0], each Bj is defined on the fixed interval [0,1]. These 
reference elements are the hat functions defined on a uniform subdivision of [0,1] 


into N subintervals. Using these reference elements, we can rewrite the 
above expressions for our matrix elements as Qy = Bj(0)Bj(0) -j- r(t)J BjBjdx, 

Hy = — I BjJDBjdx, and T- j = +(t) xBj3)Bjdx. Notice the appearance of r(t). 

J 0 Jo 

In the case of time-dependent delay, this represents a straightforward derivative of 


the unknown parameter. In the state-dependent cases, this represents the quantity 
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(l — g(w°(t)) ] ^ or n,ax ( D 8(w 0 (t))-F(t,q,w 0 (t),2w| ( (t)B k )-X , (1 — 5) ]. We now rewrite 
equation (5.1) as 


r(t)Q(t)w(t) = Hw(t) + +(t)T(t)w(t) + F(t,w) 

(5.2) 

t(0)Q(0)w(0) = C 

where H and T are constant matrices which are three-banded due to the nature of 
the linear splines, and do not depend on the unknown parameters. The matrix Q, 

also three-banded, depends on r in a simple way: Q 0 , 0 = + Bq dx , and no 

other entry of Q depends on r. 

At this point, we would like to return to two outstanding questions from 
sections 3 and 4, namely our claims that J N varies continuously in the parameters, 
and that our approximating systems have unique solutions. 

Consider N and M fixed. We show that J N is continuous on I1 M by showing 

the vector w(t) is continuous for each t. We can demonstrate this continuity by 

verifying that the differential equations (5.2) depend continuously on the 
parameters. The continuity on the parameters 4> and 7 is clear from the simple 
dependence: The parameter 7 appears only in F (recall f is continuous in 7 

by assumption) and the initial condition appears through a projection. The 

parameter r also appears in a simple (multiplicative) way except that the 
derivative r also appears. For the time-dependent case, we consider only T€T m , 
i.e., r is a linear spline. For this special type of function, |*r 1 — can be 

bounded by l-Tj — t 2 | 00 and thus, it follows directly that J N varies continuously 
with T in the C[0,T] topology. In the two cases of state-dependent delay, we 
consider the appended system, (w,f), and claim that this system of differential 
equations is continuous in g (for g varying in C[x,x]XF) in the first class of 
state-dependent delays, or in g (for g varying in W 1,0 °[x,x]) in the second class. 

The existence and uniqueness of solutions follows from the fact that the 
right hand side of equation (5.2) (modified so that w, w(0) are alone on the left) 
combined with the right hand sides of the equations for t in the cases of state- 
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dependent delay, are Lipschitz continuous in w, or (w , t). The verification of this 
continuity is straightforward. 

We now turn to some computational concerns. The time-consuming 
computations (at least on a conventional computer, as opposed to a supercomputer) 
are the numerical quadratures required for the inner product evaluations. Using 
the reference elements defined above, these quadratures can be evaluated and 
stored once for any fixed N. As we iterate on the parameters, these quadratures 
are reused. If, as is the case in the examples we present in the next section, we are 
only attempting to estimate an unknown delay function, then this method is very 
efficient (in this case, the elements of C are also computed only once for a given 
N). 

The simple dependence of the matrices Q and H on the delay was 
acknowledged in [2], and exploited in [3] (the use of the reference elements to avoid 
recomputing quadratures provides a significant computational savings in the 
estimation of constant delays as well as in the estimation of variable delays), 
however, the explicit use of these reference elements was first carefully stated and 
formalized in the context of estimation of unknown variable coefficients in partial 
differential equations in [12]. 

Focusing on the unknown r, we make some observations regarding the 
implementation with the constraint set ‘Tq. Any function has a 

M 

representation of the form r = 2*7i bj(t). Thus, the estimation of is 

equivalent to the estimation of 7? = in ]R Mtl . The constraints on r are 

readily converted to constraints on 77 . The requirement that r 0 <; T(t) <£r 
VtG[0,T] translates to r 0 <; 7^ <1 r for each i = 0,1,... ,M. The requirement that 

-M ^ t(t) <, 1-5 becomes ^ C77 i+ i — ^7*) <L ^ for eac h i — 0, 1,...,M. 

Similarly, for the estimation of a state-dependent delay of the first class, our 

M 

unknown is gGT», having the form g = 2 77 - bj(x). The corresponding constraints 

1=0 1 1 

„ . -Mo (x — x) # N /z g (x — x) 

on 77 are g L £ T} { ^ g u and — ^ ^ (77 j+1 — T]) ^ for each 

i = 0,1,... ,M. 

As noted earlier, the constraints on the parameter set for the second class of 
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state-dependent delay are not easily characterized, and therefore we cannot 
translate them into corresponding constraints on the approximations. We can, 
however, proceed with our estimation procedure without implementing all the 
constraints, and may still be successful (see, eg., example 6, below and [14]). 

As a final comment, we note that in order to prove our convergence theorems 
for both cases of state-dependent delay, we impose constraints on the approximate 
delays by multiplying the differential equations which they satisfy by \ N,M ; this is 
done in order to assure that each r N,M belongs to T c . Our convergence theorems 
then guarantee that the approximate delays, r N,M , converge to the ’’true” delay r, 
which satisfies the constraints of T c in the first instance (i.e., no externally 
imposed constraints are used). We thus conclude that, for large enough N and M, 
the constraints are not actually used, i.e., for large enough N and M, X N,M = 1, and 
therefore can be left out. This is not the case for the constraint in the second 
class of delays, used to keep t N,M ^ 1 — S. This constraint must, in theory, be 
imposed (in our numerical example, however, we have not imposed it and were still 
able to succesfully identify the parameter - this may well be due to the fact that 
our test example is ’’nice”, i.e., it has an exact solution, which may be unique). 

The translation of the constraints of T c and T g into implementable form 
within Tc and T g is clearly possible when these sets are based on linear splines. 
It is not clear how this could be done with higher order splines. For the state 
approximation, however, one could use higher order splines with few changes in the 
numerical scheme described above. The matrices would have more bands and 
evaluation of z£ would not be quite as simple, but one could still make use of 
reference elements to avoid excessive recomputations of quadratures. 


6. Numerical Test Examples. In all examples, we estimate one unknown delay, 
and in some an unknown constant appearing in the function f. We use a Levenberg- 
Marquardt algorithm (the subroutine ZXSSQ in the IMSL library) to solve the 
minimization problem (P N,M ). For each iterate of q, we solve system (5.2) using 
DGEAR (also from IMSL, a stiff ordinary differential equation solver, based on 
Gear’s method). The evaluation of J N requires only w 0 (tj). While T, and hence also 
T M carry derivative constraints, we did not impose these (this makes it easier to 
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implement the scheme for testing purpses); all of our examples work without the 
constraints, probably because they have exact solutions (and in many cases, 
solutions may well be unique). 

In each case, we haye a known ’’true” delay and a corresponding analytical 
solution to equation (2.1). We generate data by evaluating our solution at various 
time values. We choose a value for N and M and make an initial guess for r M , g M , 
or g M (and possibly 7); since we do not presume to have any knowledge of the 
, shape of the unknown functions, we always guess a constant. For a true problem, 
we might run our algorithm for many values of N and M. In most of our test 
examples, we get a very good match with N-30 and fairly low values of M. Thus, 
in many cases we only report the results for one such run. 

We present a representative sample of test problems below, chosen to 
highlight the capabilities and interesting limitations of the method discussed here. 
Further examples can be found in [14], 

Example 1. This work was motivated by problems in biology, where the time- 
varying delay changes in a periodic fashion. The delay of our first example was 
chosen to be simple (piecewise linear) but exhibits this oscillatory behavior. 

Our model equation is 

x(t) = bx(t — r(t)) — dx(t) for 0 <. t <; 1.5 

x 0 (s) = e — s for — r(0) ^ s ^ 0. • 

We have chosen the true r(t) to be the function which increases linearly from 
(t,r(t)) = (0,0.25) to (0.5, 0.5), then decreases linearly to (1.0,0.25), then increases 
again to (1.5, 0.5) (see, e.g., ’’True tau” in Figure 1A). We also have an unknown 
7 = (b,d) with true value (4.0, 2.0). An analytical solution can be found with the 
method of steps, and was evaluated at 15 t-values between 0 and 1.5 to produce the 
’’data”. 

IA. We have approximated the states with linear splines using N=*30, and the 
delay function with linear splines and M*»3. We made an initial guess of r s 0.2 and 
7 = (1.0, 3.0). Our best fit for t 30,3 is graphed in Figure 1A. We estimated 
730.3 _ (3.906, 1.941). Our residual is J 30 (p 30 ’ 3 ) = 0.360 X10* 6 . 

IB. We repeated the same example problem, but after adding random noise to 
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the data. The noisy data Xj is obtained from the original data Xj by setting 
Xj = NalXjl + Xj 

where N represents a random number from a standard normal distribution (zero mean 
and unit variance), and a is the level of noise. In this example, we chose cr = 0.025 
(i.e., we have introduced noise at a level of 2.5%). We have estimated only the 
delay, beginning with the initial guess of 0.3. The best estimate, T ' , for this 
noisy data is plotted in Figure IB. Our residual in this case is J 30 (r 30,3 ) = 
0.170 X10" 2 . 

1C. Here we have used the same example problem, with the ’’pure” data, but 
now with M=6. The choice of M“3 was a best case in the sense that the nodes in 
the true r coincided with the nodes of our approximation scheme. Clearly we have 
no hope of success for estimation of such a T if our nodes do not coincide, but a 
better test of our method uses more than enough nodes, thus allowing the 
possibility of estimating a choppier T than the actual. As can be seen from Figure 
1C, overestimating the number of nodes does not cause over-oscillation in the 
estimate for r. Our residual is J 30 (t 30,6 ) = 0.487 X10" 8 . 

Example 2. In this example, our true delay exhibits oscillatory behavior 
similar to that of Example 1, but is not piecewise linear. We first chose the ’’true” 
delay to be r(t) = | + sinj|tj, and a true solution of x(t) = 2 — (t— l) 2 . We 
then chose our model dynamics to be 

x(t) = 2x(t— r(t)) — x(t) + g(t) for 0 <; t <; 7.0 

x 0 (s) = s + 1 for — r(0) <; s <; 0 

where we compute g(t) so that the above equation holds. We evaluated our true 
solution at 15 time values for our data. We approximated the states with linear 
splines with N=30, and the delay with linear splines and M=7. We used an initial 
guess for r = 1.0. Our estimate, along with the true function and initial guess, is 
graphed in Figure 2. Our residual for this example is J 30 (t 30,7 ) = 0.141 X10 -3 . 

Example 3. In [9], [15], and [16], several examples of state-dependent delay 
differential equations, with analytical solutions are presented. We have tested our 
algorithm using modifications of these examples (in some cases we have shifted the 
time scale, in others we have rewritten their state-dependent delay as a time- 
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varying delay, etc.). One such modification (of Example 5.1 in [9]) is: 

x(t) = — p= x(t — r(t)) for 0 <; t <, 2 

2^t+l 

x 0 (s) =1 for — r(0) <; s <; 0. 

The true t is given by: 

T(t) = (t - 4t+l + ^2 for 0 ^ t <; 1 

1 |(t-l) - (1-Hi)^ft+1 + ■& for 1 <; t <; 2. 

We tried to estimate r(t), but found our optimization algorithm would never 
converge. In fact, the algorithm would match r for later values of t, but would 
wander with values of r for low values of t. In terms of our spline 

M 

approximations, r M = we would see T} u all approach values close to 

i=o 1 1 

the interpolation values for the true T, but the algorithm could not locate a ’’good” 
value for T) 0 . For all candidates for T} 0 that the algorithm tried, we would see a 
small residual. This suggests nonuniqeness. This example illustrates an important 
(yet obvious, after the fact) limitation of any parameter estimation routine. With a 
constant initial function, any r will provide a correct fit to data for early times; 
i.e., there is no unique solution to the inverse problem, in the sense that a whole 
class of functions will fit the data. Let r* represent the true r; any function f 
satisfying t — f(t) <; 0 for all t for which t — r*(t) <; 0 can match the data 
exactly, since even though (for these times) t — f(t) t — T*(t), as long as both 

values are negative, the same constant value of x is used. 

We tried the following experiment to verify the above discussion. We chose 
M-4, set 77 0 = J m t( 0) and T] l = / M r(.5), and held these fixed. We then tried to 
estimate r on the interval [.5,2], i.e., we searched for ?72»^3> an< i With initial 

guess = 0.5 for each i = 2,3,4, we were able to succesfully estimate r, with a 

residual of J 30 = 0.345 X 10' 8 . See Figure 3 for plots of T (true, initial, and 
estimated). 


Example 4. We repeat the model dynamics of the above example, however, 
having learned the problems a constant initial function creates, we modified the 
equation accordingly. Thus, our model equation is: 
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for 0 ^ t £ 2 


x(t) = — p=x(t — r(t)) + g(t) 

2it+l 

x 0 (s) = s + 1 for — r(0) ^ s ^ 0. 


We chose a true solution, then evaluated g(t) so that the above equation holds. 
Our true r, motivated by an interesting delay found in [16], is given by: 


t-(t-l ) 3 


- { iii 


for 0 £ t £ 1 
for 1 ^ t £ 2. 


-4 2 it-n 3 +i 

With data at 14 time values between 0 and 2, N-30 and M-6, and an initial guess 
for r of Tsl, we obtained the estimate pictured in Figure 4. Our residual for this 
example was J 30 (r 30 ’ 6 ) = 0.311 X10 -6 . 


Example 5. In this example we estimate a delay which is state-dependent of 
the first class discussed in the body of the paper. The model equation is: 

x(t) = lexp[— x(t — T(t))] for 0 ^ t ^ 6.0 


x 0 (s) = In ( |s + 3) 


t(t) = 1 


g(x(t)) 

g(x(t-T(t))) 


for — t(0) <; s <; 0 
for 0 <; t <; 6.0 


t ( 0 ) = r i 

where we have chosen the true parameters g(x) — exp(— x), and = 3. An 
analytical solution for this example is given by x(t) = ln^|t + 3 J and 
T(t) = |t + 3 . Our data is collected at 15 values of time between 0 and 6. We 
have estimated both and g; beginning with initial values of r° = 1.0 and 
g° = 0.1, we obtained estimated values of r? 0,3 = 3.007 and g as depicted in 
Figure 5. Our residual is J 30 (g 30,3 ) = 0.130 X10' 7 . 


Example 6. Our final example is a nonlinear delay differential equation in 
which we estimate a state-dependent delay of the second type. We use a slight 
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modification of Example 1.1 of [16]. The model equation is: 

x(t) = £^x(t)x(t — r(x(t))) for 0 <; t <; 2.3 

x 0 (s) = s + e for — r(e) <; s ^ 0 

and our true r is given by r(t) = g(x(t)) with g(x) = (e— l)lnx. We use data for 

15 values of time between 0 and 2.3. As in the previous examples, we use linear 

splines to approximate the states, with N-30. The analytical solution is given by 
(t*f g)/g 

x(t) = e . For this example, we choose [e,e+4] as the state interval and 

approximate g here using M-5. Our initial guess for g was g° = 2. In Figure 6, 
we have plotted the initial guess, and the true and approximate functions, g. Our 
residual is J 30 (g 30,5 ) = 0.275 X10' 6 . 
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